The invention relates to a method and a device for monitoring the surroundings of a vehicle.
There exist a plurality of systems for monitoring the surroundings of a vehicle. Such systems are used, for example, to implement an accident prevention (so-called precrash) system, an automatic cruise control, and to monitor the blind angle with respect to the driver's field of vision (the so-called “blind spot”). To this end, systems are used that avail themselves of various sensors. Some examples are radar sensors, LIDAR sensors, ultrasonic sensors, and video sensors.
It is known to display the images, recorded by video cameras, directly on an image display unit in the interior of the vehicle or to make slight adjustments with respect to the image content. Such adjustments relate, for example, to the brightness and the contrast of the image content.
Owing to the direct display of the image contents, it is mandatory for safety considerations that the recording optics exhibit only slight optical distortions. Only then is it guaranteed that the viewer—generally the driver of the vehicle—, can easily interpret the image content.
So-called “image grading” is used for optimizing the noise in images, provided by a camera in the visible or infrared spectral range. Owing to the image grading, which corresponds to a temporal integration or averaging of a number of individual images, the signal-to-noise ratio of the images, represented in an image display unit, can be improved. Therefore, in order to reduce the noise components in an image to be represented in the image display unit, several images are recorded by the camera and subjected to image processing, during which a number of images are processed. With the use of computer technology and/or during the digital image processing, this can also be achieved by averaging the pixel data, especially in consideration of their brightness in the respective images, and the color data, which may or may not be included. This does not have to be done necessarily pixel-by-pixel, but rather can be done by averaging the simultaneous and locally identical pixel regions of a number of images.
In order to achieve good results, the number of images should be taken from approximately the same solid angle in order to avoid loss of detail in the image to be represented in the image display unit. This requirement is fulfilled for a camera that is moving along a vehicle's longitudinal axis in a vehicle. Blurring of the image, represented in the image display unit, owing to the solid angle that has changed slightly due to the longitudinal movement is barely perceived by the driver of the vehicle.
However, the situation changes in the event of large changes in the solid angle, for example when driving along a curve. Due to the high angular speed, a camera-recorded object is always somewhat offset from image to image during the recording process. In the case of image grading, this offset leads to blurring, primarily in the edge region, and may perhaps lead to a distortion of the objects, a state that manifests itself through a ghosting effect in the image display unit. Such a deteriorated image quality can make it difficult for the viewer of the images—the driver of the vehicle—to interpret the image contents in the image display unit and, thus, leads to a decline in the driver's ability to detect important objects, such as pedestrians, bikers or animals.
German patent document DE 100 59 315 A1 discloses an arrangement for monitoring the surroundings of a vehicle with at least one wide angle view camera, one image processing unit with a mechanism for removing the camera-recorded images and one image display unit. In such an arrangement, the mechanism works in an adaptive manner on the basis of a selected section of the image and on the basis of a selected direction of gaze. The arrangement ensures that the distortions, generated by the wide angle lens, are rectified to the effect that an arbitrary two dimensional imaging of the plane of the camera image is transformed into the image plane of the display image of an image display unit. The adaptation of a selected section of the image and/or a selected direction of gaze can be performed automatically on the basis of the steering angle or the speed of the vehicle. An image optimization does not take place beyond the rectification of the image, recorded by a wide angle lens.
Therefore, the object of the present invention is to provide a device and a method for monitoring the surroundings of a vehicle. With this method and device, the image quality of the camera-recorded images may be optimized for representation.
According to the invention, a device is provided for monitoring the surroundings of a vehicle with at least one camera, one image processing unit, and one image display unit. A means for reducing the noise components in the camera-recorded images in the image display unit is operatively configured for carrying out a reduction in the noise components by image grading, during which an averaging of a number of images ensues. The number of images, processed for the image grading, is variable. The variability of the image grading enables a situation-dependent optimal image grading, so that the blurring on the edges and the distortions of the objects owing to the ghosting effect may be reliably avoided.
According to a preferred design, the means for reducing the noise components is designed for carrying out an adaptive adaptation of the number of images, processed for image grading, as a function of at least one driving dynamic parameter of the vehicle. An adaptive adaptation of the image grading (that is, the variation of the integration time and/or a change in the number of images—also called the grading depth), which is subjected to an averaging, as a function of one or more driving dynamic parameters, such as the speed, the steering wheel angle or the yaw rate, makes possible the desired optimization. The degree of optimization and, thus, the quality of the image, represented in the image display unit, may be made a function of the driving dynamic state of the vehicle. A maximum noise optimization is practical, for example, at a low speed and small steering wheel angle, as is typical for driving straight ahead in the inner-city area. In the event of cross country or freeway driving, which usually takes place at a higher speed and smaller steering wheel angle, a somewhat lower optimization is sufficient. If the noise optimization is reduced, the image processing unit has a somewhat smaller computing load. The somewhat lower image quality is acceptable in this driving situation. The least image grading is performed at a low speed and a large steering wheel angle. Such a situation corresponds, for example, to turning off into the city at a high yaw rate. This significantly reduces, as compared to conventional devices, the effect of “smearing” the image, displayed in the image display unit. Even though such a driving situation includes a certain amount of noise, the driver's detection capacity is not automatically reduced, since the human eye reacts very sensitively to relative movements.
Another practical design provides at least one sensor for capturing the driving dynamic data. The sensor is coupled to the image processing unit. In this design, the driving dynamic parameters are determined on the basis of sensor values, which are determined by the at least one sensor and are transmitted to the image processing unit. According to one design, the at least one sensor can be a speed sensor. In this case, the driving dynamic parameter is a function of the speed value, determined by the sensor. In another design, the sensor may be designed as a sensor for determining the steering wheel angle. In this case, the driving dynamic parameter is a function of the angle, determined by the sensor. In another design, the at least one sensor is a sensor for determining the yaw rate of the vehicle; and the driving dynamic parameter is a function of the yaw rate, determined by the sensor. A device, according to the invention, may exhibit any combination of these types of sensors. Even the presence of one single type of these sensors is adequate. Preferably, a combination including a speed sensor and a sensor for determining the steering wheel angle is utilized.
Another practical embodiment of the device provides that the driving dynamic parameters for influencing the number of images that are processed for image grading are computed by a computer of the image processing unit. In this design the sensor values, transmitted by the at least one sensor to the image processing unit, are received and subjected to a specified algorithm in order to determine a driving dynamic parameter that corresponds to the driving situation, in order to vary the grading depth of the image grading when such a parameter is on hand.
Another design provides that a plurality of driving dynamic parameters are stored in a storage unit of the processing unit. The parameters can be read by the image processing unit in reaction to the reception of one or more sensor values for further processing. Therefore, a matrix including the driving dynamic parameters and, for example, the speed, the steering wheel angle, and/or the yaw rate is stored in the storage unit. Once the image processing unit has received the sensor values, sent by the at least one sensor, a driving dynamic parameter that corresponds to the sensor values is read from the storage unit and used to influence the grading depth in the image grading process.
In principle, the camera that is used in the device, according to the invention, may be of any nature. It is preferred that the camera be designed as an infrared camera and/or a camera that is sensitive to visible light. The device, according to the invention, may also be designed with more than one camera. In this case, it is contemplated that the inventive device exhibits camera models that are sensitive to a number of different spectral ranges.
The same advantages, as described above in connection with the inventive device, are associated with the method, according to the invention. In the inventive method for monitoring the surroundings of a vehicle with a device having at least one camera, one image processing unit, and a means for reducing the noise components in the images recorded by the camera, and including one image display unit, the means for reducing the noise components carry out a reduction in the noise components by image grading. At the same time, the image grading includes an averaging of a number of images; and the number of images that are processed for the image grading is variable.
According to a preferred design of the method, an adaptive adaptation of the number of images that are processed for image grading ensues as a function of at least one driving dynamic parameter of the vehicle.
The speed of the vehicle, the steering wheel angle of the vehicle, and/or the yaw rate of the vehicle are considered to be the parameters. In addition, other driving dynamic parameters, which are captured by the sensors of the vehicle, may also be used for influencing the image grading.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.